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ABSTRACT: To keep pace with today's rapidly growing knowledge-driven society, productive self¬ 
regulation of one's learning processes are essential. We introduce and discuss a trace-based 
measurement protocol to measure the effects of scaffolding interventions on self-regulated 
learning (SRL) processes. It guides tracing of learners' actions in a learning environment on the fly 
and translates these data into indicators of engagement in SRL processes that reflect learners' 
use of scaffolding interventions and contingencies between those events. Graphs of users' 
learning actions in a learning environment are produced. Our trace-based protocol offers a new 
methodological approach to investigating SRL and new ways to examine factors that affect 
learners' use of self-regulatory processes in technology-enhanced learning environments. Our 
application of the protocol was described in a study about Learn-B, a learning environment for 
SRL in the workplace. The findings of the work presented in this paper indicate that future 
research can gain substantially by using learning analytics based on users' trace data and merging 
them with other quantitative and qualitative techniques for researching SRL beliefs and 
processes. 

Keywords: Self-regulated learning, micro-level process, trace-based methodologies, learning 
analytics, graph theory, learning technology 

1 INTRODUCTION 

Self-regulated learning (SRL) is acknowledged as an essential skill for lifelong learning in today's 
knowledge-driven society (Klug, Ogrin, & Keller, 2011). It is often characterized as a process in which 
learners take initiative to identify their learning goals; and choose and regulate their learning strategies, 
cognitive resources, motivation, and behaviour to optimize their learning outcomes (Boekaerts, 1997; 
Winne, 2010a; Zimmerman, 1990). Several research studies show that, in various learning contexts, 
learners often sub-optimally regulate their learning processes or simply have inadequate models of their 
learning process that leads them to misevaluating their learning (Bjork, Dunlosky, & Kornell, 2013; 
Margaryan, Milligan, & Littlejohn, 2009; Winne, 2005). 
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To address this challenge, self-regulatory scaffolds and pedagogical affordances have recently become 
of interest to researchers as a means to support learners' engagement in SRL processes (see, for 
example, Dabbagh & Kitsantas, 2005; Hadwin, Oshige, Gress, & Winne, 2010; McLoughlin & Lee, 2007; 
Zhou & Winne, 2012). Although empirical research has shown that scaffolding can foster learners' 
engagement in some of the main elements of metacognition and SRL — such as self-observation, self¬ 
reflection, or goal-orientation (Greene & Azevedo, 2010) — no methodological framework has yet been 
developed that can measure the impact of individual scaffolds, embedded tools, or different elements of 
SRL in the context in which they are used. The context also plays an important role in the effect of a 
given scaffold. Current research has shown that SRL is highly context dependent and specific features of 
a learning environment can influence whether learners engage in SRL processes and the extent of their 
engagement (Boekaerts & Cascallar, 2006; Whipp & Chiarelli, 2004; Winne, 2010a, 2010b). In most 
research studies, however, learner engagement in SRL processes is measured via self-reports that 
represent learners' perceptions of their own beliefs and abilities, statically and outside of the actual 
learning environment, rather than indicating their dynamic cognitive processes and responses in the 
immediate temporal context of scaffolding interventions. Although recent research has begun to pay 
more attention to methods for collecting data about learning as it happens through so-called trace data 
(Hadwin, Nesbit, Jamieson-Noel, Code, & Winne, 2007; Winne, 2014), more work is needed showing 
how trace data are 1) integrated into research methods and 2) aligned with theoretical models of SRL in 
which effects of technological scaffolds for SRL are studied. 

To be able to investigate whether, and to what extent, certain theorized SRL elements can be enhanced 
through scaffolds provided by software tools and interventions in a learning environment, in this paper, 
we introduce a trace-based methodology to measure use of SRL processes at both macro and micro 
levels, as well as the effect of tool components on certain theorized SRL elements. Macro-level 
processes indicate categories or phases of SRL as defined in the underlying theory such as planning, 
monitoring or evaluation, whilst micro-level processes indicate more specific activities within each of 
those phases or categories such as goal setting within the planning phase (Greene & Azevedo, 2009). In 
our research, we pursue an event-based conceptualization of SRL processes (Winne, 2014). We 
developed and applied a trace-based microanalytic measurement protocol methodology to investigate 
users' deployment of SRL processes in the authentic, dynamic context of learning. Via this methodology, 
learners' actions are captured on the fly and in the authentic context of occurrence. One of the greatest 
advantages of trace-based methodologies is that they allow for grounding analyses and inferences 
drawn from analyses on accurate and authentic data that proximally identify learning events in their 
very own context (Azevedo, Moos, Johnson, & Chauncey, 2010; Greene & Azevedo, 2010; Winne, 2010a, 
2010b; Winne & Perry, 2000; Zhou, Xu, Nesbit, & Winne, 2010). 

This paper does not aim to report on the findings of any specific study or introduce a theoretical model. 
Rather, it aims to introduce a novel measurement protocol for use in different studies that look at the 
effects of technological scaffolding interventions to support SRL. The protocol has been applied to two 
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different studies (Siadaty, Gasevic, & Hatala, 2016a, 2016b) that offer 1) theoretical details for the 
adopted SRL model and the choice and design of technological interventions, 2) hypothesized effects of 
the technological interventions on specific SRL phases, and 3) findings regarding the proposed protocol 
to both test the validity of the hypothesized effects and detect and explore which interventions had the 
strongest effects on specific SRL processes. 

We organize this paper as follows: Models conceptualizing self-regulated learning processes and various 
methods used in the literature to measure these processes are introduced in Section 0. Section 0 
describes our measurement protocol, starting with a discussion of the prerequisites for applying it, 
namely, formulating the underlying SRL model and the intended scaffolding interventions, followed by a 
detailed explanation of the steps in the protocol and examples of how we applied them in our research. 
Section 0 discusses the implications of the proposed protocol for both research and practice and Section 
0 concludes by offering directions for future research. 

2 ASSESSMENT OF SRL PROCESSES 

In this section, we provide an overview of existing perspectives and models of SRL, and describe 
methods commonly applied in current empirical research for measuring SRL according to these 
perspectives. 

2.1 Self-Regulated Learning: Conceptualization 

The concept of SRL emerged from within educational psychology research in the 1980s and became 
increasingly popular as the concept of learning to learn found its way through educational 
environments. Since then, it has been a subject of extensive study in different disciplines, such as 
training, academic education, medical education, and educational psychology (Carmen & Torres, 2004; 
Karoly, 1993; Schunk & Zimmerman, 1994; Winne & Perry, 2000; Zimmerman, 2001). 

Different models posit alternative views on how learning is self-regulated (Boekaerts, 1997; Pintrich, 
2000; Winne & Hadwin, 1998; Zimmerman & Schunk, 1989). SRL models in general aim to describe how 
learners take control of and manage their learning processes (Wolters, 2010). One way to differentiate 
these models is through different conceptualizations of SRL. One perspective offers an aptitude or 
component conceptualization, while another conceptualizes SRL in terms of events or processes (Dettori 
& Persico, 2008; Klug et al., 2011; Puustinen & Pulkkinen, 2001; Steffens, 2006; Winne, 2010b). The 
models using the component or aptitude perspective are more trait- oriented (Klug et al., 2011); they 
characterize SRL in terms of relatively stable, decontextualized individual differences (e.g., attitudes and 
beliefs) and identify cognitive, meta-cognitive, motivational, and emotional aspects of (self-regulated) 
learning. An important commonality within this category is that aptitudes, although relatively enduring, 
are considered adjustable, in that learners can be taught to develop the desired aptitudes, or transfer 
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the level or nature of an aptitude as they progress through learning events (Winne, 2010a, 2010b). On 
the other hand, the event or process perspective is more concerned with the way learners approach 
problems and apply their learning strategies (Steffens, 2006) in situ. These models view SRL as proactive, 
goal-oriented processes that learners deploy to acquire academic skills and competencies (Klug et al., 
2011; Steffens, 2006). Such processes are typically organized as a set of learning phases within which 
learners perform learning activities, e.g., tactics and strategies. These phases typically repeat during 
learning activities and may influence one another. 

Another way to differentiate the existing models is by their theoretical underpinnings (Greene & 
Azevedo, 2007; Puustinen & Pulkkinen, 2001). The models proposed by Pintrich (2000), Winne and 
Hadwin (1998), and Zimmerman and Schunk (1989) are most empirically supported in the current 
literature. While these models share some overlapping conceptualizations, discussed in Section 0, the 
biggest differential is the theoretical background in which they are grounded. 

Pintrich's (2000) and Zimmerman and Schunk's (1989) models are based on social-cognitive theory 
(Bandura, 1989), in that learning is shaped in terms of interactions between individual capacities, 
behaviours, and the environment. Although there is no universal, single definition for SRL, many recent 
works cite the definition provided by Pintrich (2000, p. 453): "on active, constructive process whereby 
learners set goals for their learning and then attempt to monitor, regulate, and control their cognition, 
motivation, and behaviour, guided and constrained by their goals and the contextual features in the 
environment." This definition reflects a goal-oriented perspective. Consistent with this perspective, in 
Zimmerman's view, "students can be described as self-regulated to the degree that they are meta- 
cognitively, motivationally, and behaviourally active participants in their own learning process" 
(Zimmerman, 2001, p. 15). Self-regulation in Zimmerman's social cognitive model is cyclic over three 
phases: forethought, performance, and self-reflection. Pintrich's framework of self-regulation is denoted 
via a four-by-four grid of phases and areas. The four phases include forethought, monitoring, control, 
and reflection. The self-regulatory activities related to each phase occur in four different areas, including 
the categories of 1) cognitive, 2) motivational and affective, 3) behavioural, and 4) contextual features of 
the environment. 

Winne and Hadwin's (1998) model, on the other hand, is inspired by information processing theory 
(IPT). In this model, SRL is identified in terms of events, and unfolds over four loosely sequenced, 
potentially recursive stages. These stages include 1) task definition, 2) goal setting and planning, 3) 
studying tactics, and 4) meta-cognitively adapting studying techniques. The acronym COPES is used in 
this model to describe an IPT-based structure, in the form of event units common within the four phases 
(Winne, Jamieson-Noel, & Muis, 2002). It stands for Conditions, Operations, Products, Evaluations, and 
Standards. Except for operations, the other four elements represent information that learners take as 
input to, or produce as output of their learning process (Greene & Azevedo, 2007). Conditions include 
internal (i.e., cognitive conditions such as beliefs, domain knowledge, and motivation) and external (i.e., 
task conditions such as instructional cues and time available) information available to a learner that 
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influences how the task will be engaged. Operations represent the cognitive processes, tactics, and 
strategies that learners perform to address the task. Products are the information generated as a result 
of operations, such as information recalled from memory. Evaluations are internal and external 
feedback about products, while standards are the criteria against which products and effectiveness of 
the operations used are monitored. 

In line with the various models of SRL in the literature, there exist different approaches for measuring 
how learning is self-regulated. In the following, we discuss the measurement methods available, and 
mostly applied, in the current literature. 

2.2 Self-Regulated Learning: Measurement 

To provide a succinct yet inclusive overview of the available SRL measurement methods, we start with 
describing the taxonomy provided by Schraw (2010). This taxonomy is built upon the four articles in a 
special issue of Educational Psychologist dedicated to the topic of measuring SRL constructs (Greene & 
Azevedo, 2010). The taxonomy represents the big picture on this topic well since these articles forefront 
challenges and issues raised by pioneer researchers in this field. 

In this taxonomy, Schraw (2010) divides applied measurement strategies into online and offline 
methods. Online methods are those to be applied during students' active learning episodes, while 
offline methods are taken before or after those episodes. Online methods can be either obtrusive or 
unobtrusive. Online, obtrusive methods require learners' "conscious attention." Such methods thus 
consume a portion of learners' "processing resources" and might interfere with their flow of cognition 
(not necessarily in a negative way). Among methods categorized under this label, think-alouds are 
perhaps the most frequently applied. Unobtrusive methods, on the other hand, are indicators that do 
not require learners' attention; they are gathered in the background. The most notable of these 
methods is trace logs. In general, traces can be any type of data collected from users' actions in a 
learning environment, such as their clicks on hyperlinks or options selected from a palette. Reading 
times and eye-tracking strategies are other examples of online, unobtrusive methods (Greene & 
Azevedo, 2010; Nesbit et al., 2006; Zhou et a I., 2010). 

Schraw (2010) categorizes offline methods into self-reported beliefs, current abilities, and expected 
performance. Self-reported beliefs represent learners' perceptions about their own beliefs and abilities. 
These reports may concern metacognition, epistemology, or self-efficacy, and might be measured in 
general or within a specific domain, e.g., self-efficacy for computer use. There exist a number of 
different self-report questionnaires in the literature that are often used in this regard (Carmen & Torres, 
2004). For instance, the Motivational Strategies for Learning Questionnaire (MSLQ) is one of the most 
widely used self-report questionnaires (Crede & Phillips, 2011; Duncan & McKeachie, 2005). This 
questionnaire includes 81 items, and aims to assess learners' motivational orientation and use of 
different learning strategies relative to a specific course or subject matter (Pintrich, Smith, Garcia, & 
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McKeachie, 1991). The Learning and Study Strategy Inventory (LASSI) is another frequently used self- 
report questionnaire designed mainly to assess the learning strategies that university students report 
using (Weinstein, Schulte, & Palmer, 1987). This questionnaire includes 77 items grouped according to 
ten scales, including attitude, motivation, time-organization, anxiety, concentration, information 
processing, selection of main ideas, use of techniques and support materials, and self-assessment and 
testing strategies. 

Current abilities are existing abilities and skills that learners bring with them to a learning task (Schraw, 
2010). These abilities might be general, such as students' intelligence or reasoning skills, or more 
domain-specific, such as their interest and past achievement in a given subject matter. Expected 
performance can be indicated through learners' pre-judgement of learning (JOL) regarding a learning 
task, as well as learners' verbal reports on their plans and intended strategies for successful learning. 
Such reports indicate how learners define and plan to carry out their learning goals, plus an articulation 
of their criteria for successful learning. 

Another useful way to examine existing measurement methods is to look at how SRL constructs are 
conceptualized in the underlying model. Winne (2010b) and Winne and Perry (2000) describe protocols 
for measuring SRL categorized by whether they measure SRL as events or as aptitudes. Inventories and 
think-alouds are the main protocols used in the existing research to measure SRL as aptitude. In self- 
report inventories learners are usually asked about some characteristic of their learning strategies (e.g., 
frequency, likelihood, or difficulty), in a loosely defined context (e.g., when you study, in this course, or 
for exams). To answer such questions, learners rely on their memories and previous learning 
experiences in similar situations. Their responses are typically limited to a pre-defined set of options, 
e.g., a Likert-scale of 1-5. In think-alouds, learners are asked to speak about thoughts and decisions as 
they engage in learning. Contrary to self-reports, think-alouds do not require learners to recall from 
memory or think in a particular way as directed by instructions in self-reports. Think alouds have 
potential to indicate dynamic aspects of SRL (Winne, Zhou, & Egan, 2010). Nonetheless, these indicators 
are learners' interpretations about their "in-action" events, and who decides what events are taken into 
account when describing their choices and decisions. As well, think-alouds could activate cognitive and 
metacognitive processes that would not be triggered otherwise. 

Unstructured interviews are another protocol used to measure SRL as aptitudes (Cleary & Zimmerman, 
2012; Winne, 2010b). They are similar to think-alouds in that learner responses are not limited to pre¬ 
defined answers but different from think-alouds (and rather similar to self-reports) in that learners are 
asked about their SRL processes after the learning session has finished, or about typical behaviour that 
they foresee using in future learning situations (see, for example, Kitsantas & Zimmerman, 2002). 

Traces (or trace-logs as phrased in Schraw's 2010 taxonomy) are the main protocol used (and suggested 
to be used) to measure SRL as events (Azevedo et al., 2010; Greene & Azevedo, 2010; Winne, 2010a, 
2010b; Winne & Perry, 2000). Traces capture learners' actions on the fly, in the authentic context where 
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they happen. They are defined as "observable indicators about cognition that students create as they 
engage with a task" (Winne & Perry, 2000, p. 551). For instance, a trace could be that the student 
highlights a text because he/she metacognitively monitors it as important, or clicks on a hyperlink based 
on a judgment that additional content would be useful at this moment. Although traces are the most 
suitable, available method to measure SRL "as the dynamic, contextual and adaptive process it is 
theorized to be" (Winne, 2010b, p. 275), they cannot and should not be considered the only method for 
measuring SRL constructs (Azevedo et a I., 2010; Winne, 2010b). Self-reports and think-alouds still 
provide researchers with valuable information on learner perceptions about SRL, intentions, beliefs, and 
some motivational constructs. As noted to earlier, the underlying model of SRL affects how it should be 
measured. Besides clarifying their underlying model and conceptualization of SRL, it is important for 
researchers to decide which SRL constructs or processes are most important to their research, and then 
use measurement protocols most appropriate to measuring those processes/constructs. 

Affordances of technology-enhanced learning environments provide opportunities to measure SRL 
processes based on learners' traces (Greene & Azevedo, 2010; Winne, 2010b; Zhou et al., 2010). 
Because traces are gathered within the learning environment and in their authentic context, they 
empower researchers to track learners' choices thoroughly and precisely, interactions with learning 
content, learning actions, and tactics applied on the fly. This method is often referred to as Trace 
Methodology in the contemporary research on measuring SRL features (Nesbit, Xu, Zhou, & Winne, 
2007; Winne & Perry, 2000; Zhou et al., 2010). Winne (Winne & Perry, 2000; Winne, 2010b) defines 
traces as "records of behaviour, a form of performance assessment, that provide grounds for inferring a 
learner's cognitive and metacognitive activities." In other words, while the cognitive and metacognitive 
states of learners might not be visible to researchers, traces are the "observable indicators about 
cognition that students create as they engage with a task" (Winne & Perry, 2000, p. 551). Unlike 
inventories and think-aloud methods (see Section 0), trace methodologies enable researchers to 
unobtrusively track learners' experiences through actual, in-action evidence of their cognitive and 
metacognitive states rather than relying on learners' perceptions of their use of these processes, 
sampled from memory (e.g., in self-reports), or the portion that they decide to share with the 
researcher (e.g., in think-aloud or unstructured interviews). 

Recently, several proposals have emerged for the use of traces (Molenaar & Jarvela, 2014; Winne, 2014) 
to study SRL based on sound theoretical principles and with a strong emphasis on advanced analysis 
methods such as process mining (Reimann, Markauskaite, & Bannert, 2014), graph theory (Hadwin et 
al., 2007), sequence mining (Zhou et al., 2010), and statistical discourse analysis (Molenaar & Chiu, 
2014). However, much less attention has been paid to developing a protocol that would guide 
researchers in measuring and analyzing SRL processes concomitantly with the effects of technological 
interventions based on trace data collected by technology-enhanced learning environments. 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 
3.0) 


189 


JOURNAL OF LEARNING ANALYTICS 


S °)LAR 


(2016). Trace-based microanalytic measurement of self-regulated learning processes. Journal of Learning Analytics, 3(1), 183-214. 
http://dx.doi.org/10.18608/jla.2016.31.ll 


3 THE MEASUREMENT PROTOCOL 


In this section, first we describe prerequisites to employing our proposed trace-based, microanalytic 
measurement protocol (summarized in Figure 1), and then provide a systematic process to assist 
researchers and practitioners in employing this protocol. We supplement each step with models from 
our previous research, illustrating comprehensive implementation and application cases of trace-based, 
microanalytic measurement protocols. 



Figure l:Trace-based microanalytic measurement protocol: 
a) the prerequisites, b) the measurement process. 

3.1 The Prerequisites 


The measurement method applied reflects how SRL is theorized. To set the stage for valid interpretation 
of measurements and generalization, the selection, development, and deployment of a measurement 
method (or a combination of methods) should align with the underpinning SRL model or theory (Greene 
& Azevedo, 2010; Klug et al., 2011; Winne & Perry, 2000; Winne, 2010b). Thus, the first prerequisite for 
the application of this protocol is a precise and plain formulation of the SRL model underlying the 
research plus the type of conceptualization pursued (e.g., event, aptitude, or both), and the constructs 
and/or processes (e.g., the planning process or motivation constructs) of interest to the researcher(s) 
and intended to be operationalized in a scaffolding intervention. Once the underlying SRL model is 
identified, the second prerequisite is to theorize how scaffolding supports processes within this model, 
which lead to formulating the research questions and hypotheses. Accordingly, the first step in applying 
the measurement protocol is to define the processes/constructs included in the SRL model at a fine¬ 
grained, micro level — see Figure l.a. This enables researchers to target highly specific self-regulatory 
beliefs and/or behaviours, supported via a theorized set of scaffolding interventions, as they occur in 
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real time through individual engagement in specific (academic/non-academic) tasks across authentic 
contexts (Cleary, 2011). 

We applied the proposed protocol in our research to investigate how a set of scaffolding interventions 
can support knowledge workers' SRL processes in the workplace (Siadaty, 2013; Siadaty, Gasevic, et al., 
2012). These interventions were tailored to their particular workplaces. In our related work, we 
provided a detailed description of the theoretical model along with the empirical and theoretical 
grounding for the SRL scaffolding interventions and how they are hypothesized to facilitated specific SRL 
processes (Siadaty et al., 2016a). 

In the following subsections, we illustrate how the above prerequisites are met in the application of the 
protocol in the theorization, design, instrumentation, and measurement of the Learn-B tool. The section 
on the SRL model offers only an example of how it could be theorized and supported with the 
functionalities of a learning tool. The detailed theoretical underpinnings of the model discussed in the 
following subsection can be found in related work (Siadaty et al., 2016a). 

3.1.1 The SRL Model 

To meet the first prerequisite, in this section we illustrate and discuss the formulation of the underlying 
SRL model applied in our research. This model consists of three phases: 1) planning, 2) engagement, and 
3) evaluation and reflection. They manifest the three common phases across the principal SRL models 
discussed in section 0, which, despite the terminology used, collectively characterize a number of phases 
proceeding from a forethought or preparatory phase, through a task performance or enactment phase, 
to a self-reflective and evaluation phase (Dettori & Persico, 2008; Puustinen & Pulkkinen, 2001; Winters, 
Greene, & Costich, 2008). The underlying SRL model applied in our research is grounded on these 
phases, as they encompass the need of contemporary knowledge workers to identify and plan their 
learning goals, apply strategies toward performing these goals, and reflect on their learning practices 
that would influence their subsequent preparatory processes. To define the fine-grained SRL processes 
in our model, a set of specific self-regulatory activities were identified per each phase based on the 
existing literature (Dettori & Persico, 2008; Greene & Azevedo, 2009; Puustinen & Pulkkinen, 2001). As 
coined by Greene & Azevedo (2009), the three common phases in our model are the "macro-level," and 
the specific activities within each are "micro-level" processes. These are discussed in the next sections. 

3.1.2 Planning 

This phase contains processes that precede acting and in particular includes activities related to 
analyzing a task at hand, setting related goals and planning strategies for achieving those goals (Dettori 
& Persico, 2008; Greene & Azevedo, 2009; Puustinen & Pulkkinen, 2001; Zimmerman, 2008). Entwined 
with task analysis, goal setting is often the premier step of a self-regulatory learning process. Learners 
vary significantly in the types and effectiveness of the goals they set for themselves (Valle et al., 2009; 
Zimmerman, 2008). Nevertheless, that there are differences in the nature and goals of learning 
processes between workplace context and formal education should be borne in mind. In educational 
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settings, learning is a goal in itself; in workplace settings, it is mostly a by-product of work and task 
performance (llleris, 2011; Ley, Kump, & Albert, 2010; Margaryan et al., 2009). Moreover, strategic 
planning is also closely associated with goal setting, whereby learners identify the appropriate strategies 
to perform in order to achieve their desired learning goals. In short, during this phase learners analyze a 
specific situation and/or identify the need to enhance their competencies, set their learning goals, select 
strategies to reach them, judge their perceived capability to reach the goals, and take the expected 
outcomes into their consideration. Table 1 describes the micro-level processes included in this phase. 

3.1.3 Engagement 

The Engagement phase refers to processes occurring during task effort. This phase facilitates a feedback 
loop (Zimmerman & Schunk, 1989) in that learners engage in their learning strategies, observe their 
performance, compare it with a standard benchmark (e.g., within the organization) or a goal, and apply 
appropriate strategy changes in order to respond to the perceived differences (Dabbagh & Kitsantas, 
2004; see Table 1 for the micro-level processes included in this phase). This phase thus describes 
learners' efforts not only to enact their plans and strategies, but also to monitor and track their ongoing 
progress toward a learning goal and apply changes in their planned strategies if need be. 

3.1.4 Evaluation and Reflection 

This phase refers to processes occurring after a task ends or a goal is achieved. During this phase, 
learners compare their self-observed performance against some standard, such as prior performance, 
another person's performance, or an absolute standard of performance (e.g., some criteria established 
within their organization's culture). In addition, learners review and reflect on their learning experience. 
One key aspect of this phase is the generation of new meta-level knowledge about the whole learning 
process, strategies, or self, which in turn affects subsequent SRL processes (Winters et al., 2008). The 
two micro-level processes included in this phase are described in Table 1. 


Table 1: Micro-level processes included in the SRL model and their descriptions (Siadaty, Gasevic, et al., 2012). 


Macro-level 
SRL process 

Micro-level SRL process 

Description 

Planning 

Task Analysis 

To get familiar with the learning context and the definition and 
requirements of a (learning) task at hand 

Goal Setting 

To explicitly set, define, or update learning goals 


Making Personal Plans 

To create plans and select strategies for achieving a set learning 
goal 

Engagement 

Working on the Task 

To consistently engage with a learning task, using tactics and 
strategies 


Applying Strategy Changes 

To revise learning strategies, or apply a change in tactics 

Evaluation & 

Reflection 

Evaluation 

Evaluating one's learning process and comparing one's work with 
the goal 

Applying Strategy Changes 

Reflecting on individual learning and sharing learning experiences 
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The theorized SRL model is supported by the scaffolding interventions described in the following 
subsection. The theoretical reasons for the use of specific scaffolds and the empirical results of their 
effects on each of the theorized SRL processes are reported elsewhere (Siadaty et al., 2016a). 

3.1.5 Scaffolding Interventions 

As alluded to earlier, existing research has shown that SRL is inherently contextual and the specific 
features of a learning environment can influence whether learners engage in SRL processes and the 
extent of their engagement (Boekaerts & Cascallar, 2006; Whipp & Chiarelli, 2004; Winne, 2010). Thus, 
to measure the effectiveness of certain scaffolding interventions on users' SRL processing, the second 
prerequisite to our proposed measurement protocol is to precisely formulate the context, i.e., how the 
designed interventions are expected to support the intended SRL processes. At a basic level, context can 
be modeled via a set of Condition ( lf)-Action (Then/Else) statements (Winne, 2010b). This model assists 
researchers to delineate the context of users' SRL processing via a theorized set of If-Then transitions, 
where If's represent the specific intervention conditions in the learning environment (i.e., input 
information to users' learning process), upon which, i.e., Then's, users could choose to operate an SRL 
action with respect to the applied SRL model. 

According to the SRL model underlying our research, we have designed and implemented a set of 
scaffolding interventions to support users' SRL processes in the workplace. In particular, the design and 
implementation of the various functionalities of these interventions were enhanced with 1) social 
embeddedness elements to support the social nature of workplace learning and 2) support for the 
harmonization of individual learning goals with those of the organization to nurture the contextual 
dimension of learning in the workplace. These interventions were developed and integrated in Learn-B, 
a learning software we developed and applied in our research. The Learn-B environment aims to assist 
knowledge workers to create, maintain, and pursue their learning goals according to their competence 
gaps. In the following, we introduce these interventions and formulate the respective contextual 
hypotheses, as required in the second pre-requisite. Figure 2 depicts these interventions and their 
theorized supporting effect on users' SRL processing within the underlying SRL model. Greater detail 
about the development and implementation of these interventions are provided in Siadaty, Jovanovic, 
et al. (2012) along with sample events from Siadaty, Gasevic, et al. (2012). 

The notion of context used in the definition of interventions builds on what Winne and Hadwin call 
international and external conditions. Conditions are not shown in the theoretical model used in the 
design of Learn-B, given in Table 1, as the model focuses on specific SRL micro-level processes. 
According to Winne and Hadwin (1998), external conditions are surrounding factors (e.g., instructional 
design and social networks) that influence learners' decisions associated with different micro-level SRL 
processes. For example, in the case of the study with Learn-B, social context was supported through 
different interventions that allow for social awareness, comparison, and co-operation in the following 
subsections. What Winne and Hadwin (1998) refer to as internal conditions (e.g., affective states, 
motivation, study skills) are not used in the studies with the Learn-B software (Siadaty et al., 2016a, 
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2016b). Data about internal conditions can be obtained through self-reported measures, physiological 
measures, think-aloud protocols, video analysis, discourse analysis, and so on (Azevedo, 2015). 

3.1.6 Usage Information 

Derived from collective knowledge, this intervention informs users of various learning resources within 
the organization and supports the functionality of Recommendation interventions (described in a 
following section). In other words, this intervention was developed to inform users of the social context 
of their organization around a particular learning resource. Hence, the respective hypothesis on the 
effect of this intervention was that it helps users better understand the social context of their 
organization regarding their learning goals, and thus supports users in the planning phase of their SRL 
processes. 



Figure 2. The theorized supporting effect of the scaffolding interventions on users' SRL processing within the 

context of the Learn-B environment. 


3.1.7 Social Wave 

The Social Wave intervention was designed to bring waves of the latest updates to knowledge workers 
about their learning goals, and the learning resources associated with each specific goal, plus updates 
from the learning activities of their colleagues whom they follow. The functionality of this intervention is 
similar to having a specific newsfeed for each specific learning goal or colleague about whom the user is 
interested in receiving updates. We hypothesized that Social Waves originating from users' learning 
resources or their followed colleagues support users in their planning and engagement processes. 
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3.1.8 Progress-o-meters 

We developed this intervention to help knowledge workers monitor their own learning progress within 
the context of their workplace. Being aware of one's progress in achieving learning goals, observing 
oneself within the social context of the organization, and comparing personal learning progress with 
colleagues provide users with the opportunity to gauge their learning strategies, apply the necessary 
changes if need be, and reflect on their learning process. The contextual hypothesis was that this 
intervention assists users with the engagement and evaluation/reflection phases. 


3.1.9 User Recommendations of Learning Goals 

This intervention enables users to recommend learning goals, along with all associated resources, to 
their colleagues. Allowing colleagues to recommend learning goals helps users with initiating their SRL 
cycles — i.e., the planning phase in particular — to perform task analysis and goal setting. Users can thus 
gain insight into how to define learning goals that target their learning needs, how such goals could be 
formulated, and how other members of the organization have approached these goals. 

3.1.10 Organizational Recommendations of Competences and Learning Paths 

This intervention informs users of the learning objectives and requirements of their organization, 
represented through a set of pre-defined and established competences, as well as recommended 
learning paths for achieving those competences. This helps learners harmonize their personal goals with 
organizational needs (Siadaty, Jovanovic, Gasevic, & Jeremic, 2010). Thus informing workers of the 
organizational competences of higher importance and relevance to them, their organizational positions, 
and level of expertise, along with paths for attaining each potential competence, helps them to identify 
their learning needs and therefore supports them in their planning. 


3.1.11 Knowledge Sharing Profiles 

Through knowledge sharing profiles, users can monitor the extent to which they share their learning 
experiences within their workplace in terms of owned learning resources, and compare their sharing 
activities with those of other users within the same group, project, or organization. This intervention 
informs users of the social context of the organization in terms of sharing and contributing to the 
collective, thus supporting users in the reflection phase. This intervention is meant to help users engage 
in the Reflection phase through increased self-awareness and social comparison. 

3.2 The Measurement Process 


Once the underlying SRL model is defined at both the macro and micro levels and the support provided 
by the scaffolding interventions is contextualized, researchers can capture user traces in a learning 
environment into log files and utilize them to measure the effect of the interventions on users' SRL 
processes/constructs of interest. We recognize three main steps within this measurement process 
(Figure l.b). A complete reference for this process of identifying SRL and intervention events can be 
found in Siadaty et al. (2016a). 
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3.2.1 Identifying SRL/Intervention events in the learning environment 

Before collecting user traces into log files in a specific learning environment, those events that represent 
the enactment of the desired SRL (micro-level) processes must be identified. Perhaps the most 
straightforward way to trace users' learning activities is to log all their actions as they interact with 
different modules and components in the learning environment. However, one of the main challenges in 
analyzing log data for measurement purposes, especially in learning environments, is the complexity and 
excessiveness of the detail captured in raw logs. Log files can contain an abundance of low level events, 
such as mouse clicks on different components, which do not necessarily correspond to the learning 
variables and constructs of interest to a researcher (Zhou et a I., 2010). To address these challenges, the 
scope of traced events should include only those that manifest the researchers' intended SRL context 
(defined via the second pre-requisite, see Section 0) and thus are of interest to their research questions 
and hypotheses. 


Accordingly, in the first step of the measurement process, researchers need to define the sets of SRL 
events and intervention events that could occur. By SRL events, we mean those events that indicate that 
the user has enacted one of the micro-level SRL processes in accordance with the underlying SRL model, 
while intervention events represent the activation or usage of any of the scaffolding interventions. To 
identify these events, researchers should start by thoroughly examining the affordances of the learning 
environment or software in terms of the envisioned SRL processes and scaffolding interventions. 


Descriptions of micro-level processes included in the underlying SRL model, which are elaborated on 
during the first pre-requisite to the measurement process (see Section 0), are used as a reference guide 
for identifying the related SRL events available in the intended learning environment. Looking up this 
reference guide, researchers should examine all the available software components and user interface 
elements per micro-level process, and decide whether and how any one of these components allows for 
enacting a particular process. For instance, we have defined the Goal Setting micro-level SRL process as 
to explicitly set, define, or update learning goals. Examining the Learn-B environment, we documented 
that users can define a new Learning Goal by "dragging and dropping an available competence to a 
new/existing learning goal" or update their learning goal by "removing a learning path from a 
competence." Sample events for each micro-level SRL process within the Learn-B environment can be 
seen in Table 2. 
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Table 2. The micro-level processes included in the underlying SRL model and sample indicator events from the 

Learn-B environment (Siadaty, Jovanovic, et al., 2012). 


Micro-level SRL process 

Sample SRL Events 1 

Task Analysis 

Clicking on different competences under duties or projects related to the user 
Searching for a keyword 

Goal Setting 

Dragging and dropping an available competence to a new/existing learning goal 
Adding a new learning path to a new or existing competence 

Making Personal Plans 

Choosing an available learning path as the path for a certain competence 
Assigning a recommended learning path as the chosen path for a competence 

Working on the Task 

Request collaboration for a competence, learning path, or learning activity 
Marking a learning goal, competence, or learning activity as "completed" 


Adding a new activity to an existing learning path 

Applying Strategy Changes 

Updating the properties of a learning activity, e.g., its name, start date, expected 
duration, visibility, rating, keywords, and user progress 


Rating a learning path, learning activity, or knowledge asset 

Evaluation 

Adding new keywords to or updating existing keywords of a learning goal, 
competence, learning path, learning activity, or knowledge asset 

Applying Strategy Changes 

Adding a comment for a competence, learning path, or learning activity 
Recommending a learning goal to a colleague 


To define the intervention events, researchers should follow an approach similar to the one above for 
SRL events. Here, the reference guide is the hypothesized effect of each intended intervention (as 
documented within the second pre-requisite; see Section 3.1 for more details). Referring to this guide, 
the researchers then scan the different forms and features of each scaffolding intervention, which are 
implemented and available in the intended software, and document the different ways that users can 
trigger a given scaffolding intervention. For instance, Usage Information is one of the interventions that 
we theorized would assist users with their planning practices. This intervention was implemented within 
the Learn-B environment via three different features: Analytics, Social Stream, and Social Stand, which 
were available for various resources such as competences, learning paths, and learning activities. 
Analytics, for example, provided users with statistical information such as the number of times that 
competences required for a specific duty were added the learning goals of other users. Thus, we 
documented that users can trigger this intervention by clicking the "Achievement tab (under Analytics) 


1 Within the scope of the research presented in Siadaty, Jovanovic, et al. (2012), user learning goals are defined in 
terms of competences. Associated with each competence comes one or more learning path(s). Each learning path 
is comprised of one or more learning activities, and leads to the attainment of a specific competence at a specific 
level. Each learning activity is accompanied by a set of metadata specifying its content, process and planning 
information (e.g. title, description, average time required, and delivery mode), and a set of knowledge assets. 
Knowledge assets can be in the form of learning contents such as documents, books, weblogs, videos, or 
presentations; or human resources such as a knowledgeable colleague who has already successfully completed this 
learning activity. Finally, a (learning) resource is the umbrella term used to refer to a competence, learning path, 
learning activity, or knowledge asset. 
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of an available competence, learning path, or learning activity"; or, by clicking "the comments tab of a 
competence, learning path, learning activity, or knowledge asset" which represented the Social Stand 
dimension of this intervention. Table 3 provides sample events for the proposed interventions 
implemented within Learn-B, as discussed in Section 3.1.2. Appendix A: SRL and Intervention Events in 
the Learn-B Environment provides the full list of all SRL and intervention events collected in our 
research. Such events need not necessarily be unique; that is, an event could be representative of one 
or more micro-level SRL processes, or one or more scaffolding interventions. For instance, we had 
considered the event "Adding a new competence to an existing learning goal" as a traceable action for 
both micro-level SRL processes of Applying Appropriate Strategy Changes and Goal Setting, categorized 
respectively under the Engagement and Planning macro-level phases. 

Table 3. The proposed scaffolding intervention and sample events from the Learn-B environment (Siadaty, 

Jovanovic, et al., 2012). 


Intervention Feature 

Sample Intervention Events 

Usage Information 

Clicking on the Achievement tab (under Analytics) of an available 
Competence, Learning Path, or Learning Activity 

Clicking on the data tab of a Competence, Learning Path, Learning 

Activity, or Knowledge Asset 


Clicking on the Social Wave tab of one's Learning Goal, Competence, 

Social Wave 

Learning Path, Learning Activity, or Knowledge Asset 

Clicking on Duties, Roles, Tasks, or Projects folder 


Clicking on the Goal-o-meter tab (under Analytics) of one's Learning 

Progress-o-meters 

Goal 

Clicking on the Progress-o-meter tab (under Analytics) of a Learning 
Activity 

User Recommendations of 

Clicking on a single Learning Goal under the Recommended Learning 

Learning Goals 

Goals folder 


Clicking on Users who are acquiring/have already acquired an available 

Organizational Recommendations of 
Competences and Learning Paths 

competence 

Clicking on a Learning Path for an available competence 

Clicking on the data tab of an available Learning Path, Learning Activity, 
or Knowledge Asset 

Knowledge Sharing Profiles 

Clicking on one's Analytics tab (the Knowledge Sharing Profiles tab is the 
only tab, so will open automatically) 


3.2.1 Translating Traces to SRL events 

Once the desired SRL and intervention events are defined, a tracking tool can be developed to capture 
and record these events in log files. However, based on the affordances of a given learning environment, 
users' actual, traceable actions are most often at a lower granularity than the desired events that 
researchers aim to track and capture. That is, the learning environment captures user interface 
elements not directly related to the "semantics" of SRL micro-level processes and interventions. Often 
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an entire sequence of traceable actions represents a single event. To fully capture all the occurrences of 
a desired event in a specific learning environment, therefore, researchers need to first identify the series 
of low level events (e.g., at the level of mouse clicks) that represent each of the desired SRL/intervention 
events (e.g., creating a new learning goal/task). The previous step of the protocol (see section 0) is 
meant to assist researchers in identifying these series. Accordingly, researchers can start by examining 
the primary paths — i.e., a sequence of actions within the learning environment — that lead to 
performing a specific event. Once the initial set of sequential actions is identified, researchers need to 
examine various use cases, such as alternate paths, for performing the desired SRL/intervention events 
to ensure that all possible actions leading to that event in the given learning environment are identified. 
A tracking tool can then be developed to capture and store these identified user actions into log files. 
After all the log files are collected, researchers need to develop a parsing algorithm to translate the 
finer-grained user actions into chunks of coarser-grained desirable events. To this end, researchers can 
code a sequence of low-level, fine-grained actions into patterns that repeat across users' logged actions. 
Next, pattern matching mechanisms such as regular expressions (Mitkov, 2003, p. 784) can be utilized to 
develop an algorithm to mine users' log files and locate the matching patterns as defined by the 
researcher, translating them into the desired SRL/intervention events. 

So far in this paper, we have focused on SRL events and described how they were defined, 
operationalized, and measured. In the following, we describe how this step of the measurement 
protocol was conducted in our research. 

The raw learning logs generated by the system during users' interactions within the Learn-B 
environment often contained two or more events per mouse click. Each event was captured as a stand¬ 
alone record with a unique identifier; some were common across various user actions, and some were 
specific to one or more affordances of the software. For instance, when users clicked on different nodes 
representing various learning resources, two basic events were always fired: one selecting the node, 
which we called a SelectNodeEvent, and one opening a tab on the right-hand side of the tool where the 
user could see more detailed info about that resource, which we called an OpenTabEvent. On the other 
hand, creating a learning goal, a user-defined learning path, learning activity, or knowledge asset were 
all represented through the Create event. Thus, the attribute eventType (shown in Figure 3) was used to 
denote the type of each event as it was recorded by the system. 

Moreover, to capture the SRL events in their authentic settings, it was important that users' actions be 
captured in their full context, containing all the events involved, as well as the detailed information 
related to each event. This additional information, for instance, included the resourceType (shown in 
Figure 3) on which the event was applied, the unique identifier to access that resource, and additional 
information such as the name of the node accessed and its hierarchical level in the tool. Accordingly, as 
users interacted with the different tools within the Learn-B environment, data were logged at the level 
of the abovementioned eventTypes, accompanied by additional information (if any), and written to 
records in the database tables set up in advance for this purpose. 
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Figure 3 shows a snapshot of the trace data, collected by the system on the fly. For instance, when a 
user added a new keyword to a knowledge asset, the system recorded that a TaggingEvent was 
performed by this specific user, on this learning resource, followed by an Edit event. Both events were 
time-stamped to the full date and time of their occurrence. Further information included type of tag 
(e.g., keyword), text of the tag itself (e.g., "documentation"), the type and title of the learning resource 
being tagged (e.g., "Knowledge Asset" and "Basics of SW documentation — standards," respectively), as 
well as the URIs (Uniform Resource Identifier, which indicates where a resource is stored) of all the 
entities involved, i.e., the user, the knowledge asset, and the user-added tag. The blue box in Figure 3 
represents this example. 
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Figure 3. A snapshot of the log files generated within the Learn-B environment. 

To identify patterns indicating occurrence of the intended SRL events within the collected trace data, in 
the second step of the measurement process, we developed a pattern library. This library consisted of 
patterns of sequential event types corresponding to each of the previously defined SRL events. We 
systematically examined the Learn-B environment to identify these patterns, that is, sequences of 
lower-level events triggered, along with their specific details, and captured by the tracking tool when 
users performed each of the SRL events (see Table 2). This included running each of the micro-level SRL 
processes via their indicator events as defined in the previous step, and recording each triggered event. 
For instance, one of the identified SRL events for task analysis micro-level process was "exploring 
competences included in other colleagues' learning goals." Performing this single action triggered three 
low-level events in the environment: 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 
3.0) 


200 









JOURNAL OF LEARNING ANALYTICS 


S8LAR 

SOCIETY (or LEARNING 
ANALYTICS RESEARCH 

(2016). Trace-based microanalytic measurement of self-regulated learning processes. Journal of Learning Analytics, 3(1), 183-214. 
http://dx.doi.org/10.18608/jla.2016.31.ll 

1) An OpenTabEvent could have been fired, depending on whether users clicked on the Users 
tab in the Learn-B interface or directly approached a specific competence 

2) A SelectNodeEvent (clicking on the desired competence) would happen 

3) An OpenTabEvent was fired to provide more information about that competence 


Figure 4.a shows these three low-level events and the detailed information specific to each of them. 


Pattern specification for SRL event: 
Exploring other Colleagues' Competences 


[OpenTabEvent]* 


■ tabName: “Users" 

- nodeTitle: Colleague's name 

f nodeRelName: "PERSON" 
nodeView: "target" 
serviceName: “LPC" 

- nodeid: a pettem of 2 digits, followed by a dot; e.g. 1.1. 
• resourceType: "User” 


*; Optional 



Pattern implemented in REGEX 

User .Colleague. Competence. click- 
*•*?(\w+[a-zO-9]•)(?:\s+) 

(?:(\d{2}Ad{2}/\d(4)\s\d+:\d(2}:\d{2})(?:\s+)(\d+)(?:\s+)(OpenTabEvent)( 
?: \s+) (?:tabName) (?:\s+) (?:Users) <?:\s+) (User) (?:\s+) (?:\1) <?:\s+) <?:\2) ( 
?:\s+)(?:\3) (?:\s*) (?:\4) (?:\s+) (?:nodeTltle)<?:\s*) (?:.*?) <?:\s~) (?:\5) ( 
?:\s+)(?:\1)(?:\s+)(?:\2)(?:\s+)(?:\3)(?:\s+)(?:\4)(?:\s+)(?:nodeRelName) 
(?:\s+) (?:PERSON) (?:\s+) (?:\5) (?:\s+) (?:\1) (?:\s+) (?:\2) (?:\s+) (?:\3) <?:\ 
s+) (?:\4) (?:\s+) (?:nodeView) (?:\s+) (?:target)(?:\s*) (?:\5) (?:\s+) (?:\1)(? 
:\s+) (?:\2)(?:\s+) (?:\3)(?:\s+) (?:\4) (?:\s+) (?:serviceName) <?:\s+) (?:LPC) 
(?:Ns-*-) (?:\5) <?:\s+) (?:\1) (?:\s+) (?:\2) (?:\s+) (?:\3) (?:\s+) (?:\4) (?:\s+) ( 
?:nodeid) (?:\s*) (?:(?:\d+\.)<2>) (?:\s+) (?:\5) (?:\s*) (?:\1) (?:\s+)) 
?(\d{2}/\d{2}/\d{4}\3\d+:\d<2>:\d{2})(?:\s+)(\d+)(?:\s+)(SelectNodeEvent) 
(?:\S-+-) (?:r.odeTitle) (?:\s+) (?:.•?) (?:\s+) (TargecConpetence) (?:\s+) (?:\1) ( 
?:\s+)(?:\2|\6)(?:\s+)(?:\3|\7)(?:\s*)(?:\4|\8)(?:\s+)(?:nodeRelName)(?:\ 
s+) (?:COMPETENCE) (?:\s+) (?:\5|\9)(?:\s+) (?:\1) (?:\s+) (?:\2|\6) (?:\s+) (?:\ 
3|\7) (?:\s+) (?:\4|\8) (?:\s+) (?:nodeView) (?:\s«-) (shared) (?:\s+) (?:\5|\9) (? 
:\s+) (?:\1) (?:\s-*) (?:\2|\6) <?:\s+) (?:\3|\7) (?:\s+) (?:\4|\8) <?:\s>e) <?:node 
id) (?:\s+) ((?:\d+\.)+) (?:\s+) (?:\5|\9)(?:\s+)(?:\1) (?:\s+) 
(\d{2}/\d{2}/\d{4}\s\d+:\d{2) :\d{2>) (?:\s+) (\d+) (?:\s-*-) (OpenTabEvent) (?:\ 
s+) (?:tabName) (?:\s+) (?rCompeter.ee Form Data) (?:\s-») (?:\5|\9) 

(?:\s+(?:M) (?:\s-r) (?:\8|\12) (?:\s+) (?:\9|\13) (?:\s+) (?:\10|\14) (?:\s+) (? 
:nodeTitle) (?:\s+) (?:.*?) (?:\s+) (?:\5|\9) (?:\s+) (?:\1) (?:\s+) (?:\8|\12) (? 
:\s+) (?:\9|\13) (?:\s+) (?:\10|\14) (?:\s+) (?:nodeRelName) (?:\s+) <?:COMPETEH 
CE) (?:\s+) (?:\5|\9)(?:\s+)(?:\1)(?:\s+)(?:\8|\12)(?:\s+)<?:\9|\13)(?:\s+) 
(?:\10|\14)<?:\s+) (?:nodeView) (?:\s+) <?:\6|\10) (?:\s+) (?:\S|\9) (?:\s^) (?: 
\1) (?:\s+) (?:\8|\12) (?t\a+) (?:\9|\13) <?:\s+) (?:\10|\H) (?:\s+) (?:serviceN 
ane) (?:\s+) (?:LPC) (?:\s+) (?:\S|\9) <?:\s+) (?:\1) (?:\s+) (?:\8|\12) (?:\s+) (? 
:\9|\13)(?:\s+)(?:\10|\14)(?:\s+)(?:nodeid)(?:\s+)(?:\7|\ll)(?:\s+)(?:\5| 
\9)S 



Figure 4.A sample pattern in the pattern library, denoting the SRL event "exploring competences included in 
other colleagues' learning goals": a) pattern specification, b) pattern implemented using regular expressions. 

To implement the parsing process, we developed an analyzer module called Log Parser. This module 
received users' raw log files as input and matched them against the patterns defined in the pattern 
library — see Figure l.b.ll. The input was in the form of textual collections of trace records across 
various learning sessions, retrieved from the respective databases and aggregated into one concrete log 
file per user (a portion of such log files is shown in Figure 3). The patterns were defined in terms of 
regular expressions. Regular expressions, or "regex" in short, are basically strings of characters which 
denote a pattern and are often used to find a particular text, replace it with other text or validate a 
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given input (Habibi, 2004). Figure 4b shows the regex implementation of the three-event pattern for the 
SRL event exploring other colleagues' competences as shown on the left of Figure 4. 2 


The pattern-matching algorithm in the Log Parser was implemented to first search for occurrences of all 
available patterns defined in the pattern library in users' log files, and then replace the matching event 
records with a single record indicator of the occurrence when a successful match was found. Thus, the 
output of the Log Parser was a new version of each user's log file in that extraneous events that did not 
have a matching pattern in the pattern library were removed from user's traces, and the output file 
contained only coarser-grained SRL (and Intervention) events aggregated from users' lower-level 
traceable actions. Figure 5 shows a portion of a user's raw log file as the input to the Log Parser (a), and 
the evenf-ized log file as the output of this module (b). As can be seen in this figure, the series of lower- 
level traces generated for the event record 15313, for example, are translated into the intervention 
event User clicking on an Activity. This represents the intervention feature "Organizational 
Recommendations of Competences and Learning Paths." The traces for event record 15315 are then 
translated into the SRL event User clicking on a colleague's competences, which represents the micro¬ 
level SRL process "Task Analysis." 
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Figure 5. A sample user's a) raw log file, b) event-ized log file as the output of the Log Parser. The blue box 
shows the traceable records related to the SRL event in Figure 4. 


Tools for pretty printing of regular expressions are available, but their discussion is beyond the scope of this 
paper. Some relevant discussion can be found, for example, at http://stackoverflow.com/questions/5312301/is- 
there-some-good-visual-regular-expression-editor 
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3.2.2 Creating Contingency records and Transition graphs 

Translating users' raw traces into event-ized log files as described in the previous section (and depicted 
in Figure l.b.ll) allows for operationalizing the event-based conceptualization of SRL in terms of an 
event's occurrence (Azevedo et al., 2010; Winne, 2010b; Winne & Perry, 2000). An occurrence provides a 
window for the researcher to observe the evidence (or product) of user cognition operations. Hence, an 
occurrence is merely a tally of an observable state and does not convey any information about the 
context. As such, occurrences allow for performing frequency counts of users' engagement in SRL 
processes, but they fail to capture transitions between those processes. Such transitions can provide a 
fuller picture of the contextual, dynamic nature of self-regulated learning, especially in today's 
technology-enhanced learning environments (Winne, 2010a). 

To include elements of context in the trace-based microanalysis measurement of SRL processes, we 
need to explore the contingencies between users' actions, and build the transition graphs of these 
contingencies in the third step of the measurement process. This allows for going beyond simple 
frequency counts of actions and occurrences and instead probing into the context of such occurrences, 
such as the states preceding a subsequent SRL event. Contingencies in general show what subsequent 
event was preceded by which prior event. Accordingly, they include some features of context in them 
and operationalize the event-based approach to measuring SRL at a higher level than occurrences. A 
contingency can be modelled in terms of a conditional if-then (condition-action) relationship (Winne & 
Perry, 2000; Winne, 2010b), where a set of if-then transitions can be used to represent the context of 
users' SRL processing as discussed earlier in Section 3.1. For example, when it is observed in the trace 
data of a given user that he/she "includes a specific competence in their learning goals" immediately 
after "knowing about other users' comments about that competence," the former event represents the 
condition, and the latter demonstrates the action. Here, the condition is a learning event indicating use 
of the respective scaffolding intervention, while the action is an SRL event indicative of the goal-setting 
micro-level process. For example, if out of 20 competences included in a user's learning goals, 12 were 
included immediately after viewing other users' comments, we can describe a conditional probability 
that this user metacognitively considers collective intelligence important when creating his/her learning 
goals (Prfothers comments | included competence] = 12/20 or 0.6). 

In addition to the contingencies between learners' enactment of SRL events (and other relevant events, 
such as scaffolding Intervention events), researchers can build transition graphs of users' learning 
actions and examine the predominant transition patterns. Transition graphs illustrate users' navigation 
between various events, e.g., using some scaffolding Interventions and performing SRL processes. 
Moreover, they can reveal a variety of quantitative measures adapted from graph theory, which could 
be used in describing features of SRL processes. Centrality metrics, such as degree, betweenness, 
closeness, and eigenvector centrality, can be used to denote the relative importance of a node within a 
graph (Borgatti, Mehra, Brass, & Labianca, 2009; Landherr, Friedl, & Heidemann, 2010; Yan & Ding, 
2009). For example, centrality metrics enable us to look at the more influential processes across users' 
learning sessions, or explore useful patterns such as strategies more practiced by users with a common 
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background, organizational position, or project team. A detailed discussion of the use of graph statistics 
for analysis of different aspects of SRL is provided by Hadwin et al. (2007). 


To mine the existing contingencies and build the transition graphs in the third step of the measurement 
process, researchers need to perform another level of pattern matching on users' event-ized log files. 
The objective of this step is to translate all indicator events, i.e., those included in the pattern library, to 
their respective SRL (and Intervention) events. Hence, the pattern library could be extended with 
additional sets of higher level, more general patterns that show which patterns manifest engagement in, 
or usage of which SRL or Intervention event. For example, Figure 6.a lists the textual description of the 
set of SRL events that show the Working on The Task micro-level SRL process, as described in Appendix 
A. Figure 6b shows how this micro-level SRL pattern can also be defined in terms of a regular expression 
containing the set of its representative SRL events, added to the extended pattern library. The blue box 
in this figure shows one of these SRL events generated from a user's low-level mouse clicks into their 
event-ized log file, as shown in Figure 5. 


Events indicator of the SRL Micro-level Process: Task Analysis 


Micro-Level SRL Process 

SRL Events in Learn-B 

Task Analysis 

Clicking on Duties. Roles. Tasks or Projects folders 

Clicking on a single Duty under the Duties folder 

Clicking on a single Role under the Roles folder 

Clicking on single Task under the Tasks folder 

Clicking on single Project under the Projects folder 

Clickinq on different Competences related to a Duty Role Task or Proiect 


Selecting Competences from other Colleagues' learning goals 

^ 7 ^ 

Searching fora keyword 



Pattern implemented in REGEX 

SRL.Planning.TaskAnalysis= 

~(\&+[a-zO-9]*)(?:\s+)(\d{2}/\d{2}/\d{4}\s\d+:\cl{2}:\d{2>)(?:\s+)(\d+)(?:\s+) 
(Available.DutiesFolder.click| 

Available.DuciesFolder.Duty.click| 

Available.Competence.click| 

Available.Competence.DataTab.click| 

User.Colleague.Competence.click|User.Keyword.Search)$ 



Figure 6. A pattern in the extended pattern library that denotes ail of the SRL events indicative of the "Working 
on the Task" micro-level SRL process: a) list of the indicative SRL events as described in Appendix A; b) the 

pattern implemented in REGEX. 

It is important to note that depending on the underlying SRL model, each intervention or SRL event 
could be an occurrence indicator for one or more of the researcher's intended micro-level SRL 
processes. In the third step of the measurement process, hence, when translating the SRL/intervention 
events to higher-level micro-level SRL processes, each occurrence of an event should be translated into 
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as many micro-level processes as it is defined to be representing (see Section 0). For example, in our 
research within the Learn-B environment, the SRL event "adding a new learning object (e.g., uploading a 
document or a bookmark to a web page) to an existing learning activity" was defined as an SRL event 
indicative of three different micro-level SRL processes: 1) Goal Setting, 2) Making Personal Plans (both 
related to the planning process), and 3) Working on the Task — related to the engagement process (see 
Appendix A). Accordingly, each instance of the event pattern "User.Activity.AddNewAsset" in a user's 
log file was translated into all three SRL events (see Figure 7). 


Contingency records, built from user data, demonstrate the sequential stream of a user's actions as the 
users interact with different entities in a learning environment. In the last step of the measurement 
process, a graph analysis tool can be used to build the transition graphs out of users' contingency 
records. Once the transition graphs are generated, graph analysis techniques can be applied to elucidate 
graph statistics and study users' SRL processes and use of scaffolding interventions within a given 
learning environment. 
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Figure 7: A sample user's a) event-ized log file, b) event-ized log file translated into contingency records. The 
blue boxes show how one event pattern could be indicative of, and thus translated into, two or more 

SRL/Intervention events. 


In our research within the Learn-B environment, we used the Gephi open source software (Bastian, 
Heymann, & Jacomy, 2009) to build the transition graphs from users' contingency records, and analyze 
the generated graphs in order to paint a fuller picture of how the proposed scaffolding interventions 
and/or engaged in SRL processes were used. The Gephi tool supports multiple file formats for importing 
the graph file, including CSV, GDF, and GEXF. We used the CSV file format to store users' trace data in 
terms of nodes and edges in a graph, a format that could then be imported into the Gephi tool to 
generate the transition graphs. Here, a transition graph shows conditional contingencies, where nodes 
represent user actions — i.e., performing SRL or Intervention events within the Learn-B environment — 
and each directional link between nodes represents a transition between two events. 
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Figure 8 shows a sample transition graph generated from a user's trace data. For a less cluttered visual 
representation, only the links of interest to our research question, 3 i.e., those directed from an 
Intervention event to other nodes (either SRL or Intervention nodes), are shown in this graph. The bigger 
the size of a node, the more influential in users' learning events, and the thicker a link, the more 
frequent that contingency has appeared in a user's parsed trace data. To investigate our research 
question, we calculated the graph theoretic centrality measures (Bastian et al., 2009) for each proposed 
scaffolding intervention within the graph of user learning actions. Centrality denotes the relative 
importance of a node within a graph and could be identified via degree, betweenness, closeness, or 
eigenvector centrality, the most commonly used centrality measures in various domains (Borgatti et al., 
2009; Freeman, Roeder, & Mulholland, 1979; Landherr et al., 2010; Yan & Ding, 2009). These graph 
statistics can be used to reveal the relative importance of researchers' desired interventions within a 
network of user learning actions. 



Figure 8: A sample transition graph from a user's log file translated into a time-stamped sequence of 
contingencies. Only the links starting at the Intervention nodes are shown in this graph. The size of a node 
indicates its influence in the graph; the thickness of a link shows how frequently that link occurred in user trace 
data. 


In one of our relevant research questions, we were particularly interested in investigating the most effective 
scaffolding interventions (developed within the Learn-B environment) in supporting users' SRL processes within 
the Learn-B environment. 
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In the context of the Learn-B environment, we were interested to find out whether (and to what extent) 
usage of the proposed intervention could account for engagement in SRL processes within this 
environment. Accordingly, in the context of our research, centrality was considered to represent the 
importance of an intervention or SRL event within the network of user learning actions in the Learn-B 
environment. For instance, intervention events with higher degrees could indicate a variety of different 
usages in the learning process. Intervention nodes with higher closeness values indicate that users could 
easily perform their SRL practices or use the other interventions. High betweenness values specify those 
interventions used as a bridge to perform SRL practices or other interventions. Intervention events with 
higher eigenvalue centralities denote those used before/after other well-performed events. In addition 
to centrality metrics, graph statistics can be correlated with more variables collected from users' 
learning actions or via other instruments, such as self-reports, to study the effect of those cofounding 
variables (such as users' computer skills or motivational strategies) on users' SRL processes. For 
example, confounding variables such as general computer skills or familiarity with organizational 
responsibilities, collected via self-reports, could potentially affect the frequency of user engagement in 
certain SRL processes, which could be identified via the calculated centrality metrics. 

In this paper, we do not report on the findings of the applications of this protocol in the context of the 
Learn-B environment, which instead are covered in Siadaty et al. (2016a, 2016b). We only summarize 
the main results here. As shown in Figure 8, through its central position and also confirmed through 
future multiple regression analyses, we have found that Social Wave intervention accounts for 68% of 
the variance in users' total frequency of performing all the SRL processes considered in our study (c.f. 
Table 1). This intervention was followed by another that offered system-generated recommendations 
about learning paths, learning activities, and knowledge assets to stimulate engagement in micro-level 
processes within the forethought or preparatory phase of SRL. Our analysis of covariate influence, such 
as users' computer skills or experience in their organizational positions, detected insignificant effects. 
Siadaty et al. (2016a, 2016b) indicate that both the social and the organizational contexts should be 
taken into account when tailoring SRL interventions to support the forethought and engagement 
phases. 

4 IMPLICATIONS FOR RESEARCH AND PRACTICE 

Although the elements of our research strategy and the proposed measurement protocol are not 
completely new, together they suggest, in our view, a new, unique, forward-looking approach for 
designing scaffolding interventions and measuring their support for SRL processes in technology- 
enhanced learning environments. Self-regulated learning has become an essential skill in today's 
knowledge-driven, rapidly changing society where individuals increasingly opt in to informal learning 
settings and hence need to self-manage their learning processes. However, as research on learning and 
metacognitive processes suggests, most learners are prone to assumptions and beliefs that can impair 
their effectiveness as self-managed learners (Bjork et al., 2013); in other words, they simply do not know 
how to learn (Margaryan, Milligan, & Littlejohn, 2009). Self-regulatory interventions can thus assist 
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learners to define and manage their own learning processes. An important implication of this protocol is 
that it guides researchers to investigate the effectiveness of their desired SRL interventions grounded in 
an explicit theoretical framework and in the authentic context of their application. Our proposed 
protocol can be applied in any technology-enhanced learning environment where the authenticity of 
users' learning actions in their real context is important to researchers. Linking practice and research, 
the proposed protocol guides researchers to 1) formulate their hypotheses regarding the role of each 
designed intervention with regard to the underlying theoretical model and 2) avoid developing 
interventions isolated from real practice by designing, implementing, and aligning the integrity, 
effectiveness, and measurement of those SRL interventions with the nature of the actual learning 
environment. 

Another direct implication of the proposed measurement protocol for both research and practice is that, 
through its prerequisites, it guides researchers to examine the provided support for SRL processes in 
their entirety, including those related to all SRL phases as defined in the underlying SRL model. Having 
reviewed the existing literature (e.g., Dettori & Persico, 2008; Greene & Azevedo, 2009; Puustinen & 
Pulkkinen, 2001; Sitzmann & Ely, 2011), we have articulated a set of more generic processes within the 
three phases of our underlying SRL model as macro-level processes, and defined the specific activities 
within each of these phases as micro-level SRL processes. These micro- and macro-level processes can 
be (re-)used as a guide by other researchers to build their very own underlying SRL model. Accordingly, 
examining the effect of the provided support at the level of the defined micro-processes enables 
researchers to provide a more accurate picture of the role of their intended interventions within the 
larger construct of SRL. 

Because SRL processes are dynamic and contextual, the proposed protocol pursues an event-based 
conceptualization of those processes and aims to measure them as a sequence of events (traces) in the 
real context where they happen. Pioneered by Winne and associates, tracing methodology has started 
to find its way as another method for examining self-regulated learning processes in formal, educational 
settings (Hadwin et al., 2007; Winne & Jamieson-Noel, 2002; Zhou & Winne, 2012). Compared to 
questionnaires, trace data are not bonded to a certain point in time, and holistically operationalize 
"what users do as they do it" (Winne, 2010a, 2010b). In our view, the trace-based methodology, the 
core of the proposed measurement protocol, together with the microanalytical measurement method, 
provide a distinctive lens through which researchers can accurately measure and analyze how learners' 
SRL processes are supported by the intended interventions. Comparable with the potential objective of 
microanalytical protocols in formal education (Cleary, Callan, & Zimmerman, 2012), this combination of 
trace-based methodology and microanalytical measurement can guide researchers in intervention 
planning and development for various technology-enhanced learning environments. 

Some practical implications are also brought forth by the proposed protocol. First, application of the 
proposed measurement protocol requires researchers to design and develop their intended scaffolding 
interventions in terms of software and log users' learning actions indicative of their use of SRL 
processes/scaffolding interventions. This would require the research team to have access to 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 
3.0) 


208 


JOURNAL OF LEARNING ANALYTICS 


S8LAR 

SOCIETY (or LEARNING 
ANALYTICS RESEARCH 

(2016). Trace-based microanalytic measurement of self-regulated learning processes. Journal of Learning Analytics, 3(1), 183-214. 
http://dx.doi.org/10.18608/jla.2016.31.ll 


programmers to implement the intended interventions for them, as well as to build particular "hooks" 
into the intended learning environments to collect and trace users' generated SRL/intervention events 
(see Section 0). Second, the application of the protocol can be additionally costly, as implementation of 
the parsing algorithms for the SRL/intervention patterns requires resources for development, and given 
the contextual nature of SRL and the potential use of different software tools in different learning 
environments, such parsing libraries may not be easily transferable from one environment to another. It 
should be noted that the above implications, however, do not originate from our proposed 
measurement protocol; rather they are inherent to a need for effectively supporting the collection and 
analysis of trace data generated through software systems that aim to scaffold SRL processes for users. 

5 FUTURE DIRECTIONS 

The proposed trace-based, microanalytic measurement protocol is built upon the event-based 
conceptualization of SRL used to capture knowledge workers' SRL processes accurately on the fly and in 
their authentic context. As discussed earlier, any applied SRL measurement methodology should be in 
accordance with the underlying conceptualization (Greene & Azevedo, 2010). In future research, this 
methodology can be complemented with other forms of data that conceptualize SRL as an aptitude 
(Winne & Perry, 2000; Winne, 2010b). Although traces can provide researches with detailed, valuable 
information on users' learning activities "in action," they are subject to some limitations and biases, and 
thus, not inherently the best or only method for gathering data (Winne et al., 2010a, 2010b). Aptitudes, 
most commonly measured via self-reports, are also essential to researching SRL, as they represent what 
users have "in mind" when they engage in SRL processes. Together, traces and self-reports can be used 
to paint a much fuller and more detailed picture of users' actual engagement in SRL processes. Although 
self-reported measures can be used as an intrinsic part of learning environments to measure some other 
motivational constructs — for example, goal orientation in academic settings (Zhou & Winne, 2012) or 
perceived usefulness of the scaffolding — in our research within the Learn-B environment, 
measurements of some important SRL-based "aptitudes" can be done as by-products of such self- 
reports. In our future research, we aim to investigate the effective combinations and accordingly 
elucidate the best research practices in combining our proposed measurement protocol with other 
methods and instruments, such as self-reports or think-alouds, which could connect our proposed 
protocol with the work suggested by Bannert, Reimann, & Sonnenberg (2014). 

Additionally, we recommend that future research investigate the extent to which different tools, aimed 
to provide scaffolding support, can support SRL processes. SRL is contextual; each software or tool 
delivers different cognitive affordances, according to which different traces of users' SRL activities could 
be expected and captured. These issues point to a need to design and evaluate how more mature, 
different types of tools can support different macro- and micro-level SRL processes. Methodologically, 
other modelling methods can be used instead of or in combination with graph theory (e.g., cluster 
analysis, hidden Markov models, sequence, and process mining algorithms). Questions to probe include 
1) what common strategies learners follow when engaging in specific SRL micro-level processes, 2) what 
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is the probability of changing one learning strategy for another under the influence of specific SRL 
technological scaffolding interventions, and 3) what types of processes are mutually exclusive with 
different process-mining algorithms. 
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APPENDIX A: SRL AND INTERVENTION EVENTS IN THE LEARN-B 
ENVIRONMENT 


SRL Events 

Macro-Level SRL Process: Planning 

Micro-Level SRL Process 

SRL Events in Learn-B 

Task/Analysis 

Clicking on Duties, Roles, Tasks or Projects folders 

Clicking on a single Duty under the Duties folder 

Clicking on a single Role under the Roles folder 

Clicking on single Task under the Tasks folder 

Clicking on single Project under the Projects folder 

Clicking on different Competences related to a Duty, Role, Task or Project 

Exploring competences included in other colleagues' learning goals 

Searching for a keyword 

Goal Setting 

Creating a new goal 

Dragging and dropping an available competence to a new or an existing 
learning goal 

Adding a new Competence to a new or an existing learning goal 

Adding a new Learning Path to a new or an existing competence 

Adding a new Learning Activity to a new or an existing learning path 

Adding a new Knowledge Asset to a new or an existing learning activity 

Removing a Competence from a learning goal 

Deleting a Learning Path from a competence 

Removing a Learning Activity from a learning path 

Removing a Knowledge Asset from an learning activity 

Setting the properties of a Learning Goal e.g. its name, deadline, visibility, 
priority, keywords and user's progress 

Setting the properties of a Competence, e.g. its name, deadline, visibility, 
current user's level, desired level, keywords and user's progress 

Setting the properties of a Learning Path, e.g. its name, expected duration, 
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visibility, rating, keywords and user's progress 

Setting the properties of a Learning Activity, e.g. its name, start date, 
expected duration, visibility, rating, keywords and user's progress 

Setting the properties of a Knowledge Asset, e.g. its name, URL, expected 
duration, visibility, rating, keywords and user's progress 

Sharing a Learning Goal with a recommended colleague 

Requesting collaboration for a Competence, Learning Activity or a Knowledge 

Asset 

Making Personal Plans 

Requesting collaboration for a Competence, Learning Activity or a Knowledge 

Asset 

Assigning a recommended Learning Path as the chosen path for a 

competence 

Requesting collaboration for a Competence, Learning Activity or a Knowledge 

Asset 

Adding a new Learning Path to a new or an existing competence 

Adding a new Learning Activity to a new or an existing learning path 

Adding a new Knowledge Asset to a new or an existing learning activity 

Removing a Competence from a learning goal 

Removing a sub-Competence from an upper competence 

Removing a Learning Path from a competence 

Removing a Learning Activity from a learning path 

Removing a Knowledge Asset from an learning activity 

Setting the properties of a Learning Path, e.g. its name, expected duration, 
visibility, rating, keywords and user's progress 

Setting the properties of a Learning Activity, e.g. its name, start date, 
expected duration, visibility, rating, keywords and user's progress 

Setting the properties of a Knowledge Asset, e.g. its name, URL, expected 
duration, visibility, rating, keywords and user's progress 

Macro-Level SRL Process: Engagement 

Micro-Level SRL Process 

SRL Events in Learn-B 

Working on the Task 

Assigning a recommended Learning Path as the chosen path for a 

competence 

Requesting collaboration for a Competence, Learning Activity or a Knowledge 


ISSN 1929-7750 (online). The Journal of Learning Analytics works under a Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 
3.0) 


216 



























JOURNAL OF LEARNING ANALYTICS 


S8LAR 

SOCIETY (or LEARNING 
ANALYTICS RESEARCH 

(2016). Trace-based microanalytic measurement of self-regulated learning processes. Journal of Learning Analytics, 3(1), 183-214. 
http://dx.doi.org/10.18608/jla.2016.31.ll 



Asset 

Marking a Competence as "favourite" 

Following a Competence 

Sharing a Learning Goal with a recommended colleague 

Recommending a Learning Goal to a colleague 

Searching for a keyword 

Marking a Learning Goal, Competence, or Learning Activity as "completed" 

Leaving a comment for a Competence, Learning Path, Learning Activity or 
Knowledge Asset 

Updating the properties of a Learning Goal e.g. its name, deadline, visibility, 
priority, keywords and user's progress 

Updating the properties of a Competence, e.g. its name, deadline, visibility, 
current user's level, desired level, keywords and user's progress 

Updating the properties of a Learning Path, e.g. its name, expected duration, 
visibility, rating, keywords and user's progress 

Updating the properties of a Learning Activity, e.g. its name, start date, 
expected duration, visibility, rating, keywords and user's progress 

Updating the properties of a Knowledge Asset, e.g. its name, URL, expected 
duration, visibility, rating, keywords and user's progress 

Following a colleague 

Creating a learning group for a Competence 

Applying appropriate Strategy 
Changes 

Adding a new Competence to an existing learning goal 

Adding a new sub-Competence to an existing competence 

Updating the properties of a Learning Goal e.g. its name, deadline, visibility, 
priority, keywords and user's progress 

Updating the properties of a Competence, e.g. its name, deadline, visibility, 
current user's level, desired level, keywords and user's progress 

Updating the properties of a Learning Path, e.g. its name, expected duration, 
visibility, rating, keywords and user's progress 

Updating the properties of a Learning Activity, e.g. its name, start date, 
expected duration, visibility, rating, keywords and user's progress 

Updating the properties of a Knowledge Asset, e.g. its name, URL, expected 
duration, visibility, rating, keywords and user's progress 
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Removing a Competence from a learning goal 

Removing a sub-Competence from an upper competence 

Following or unfollowing a competence 

Requesting collaboration for a Competence, Learning Activity or a Knowledge 

Asset 

Adding a new Learning Activity to an existing learning path 

Adding a new Knowledge Asset to an existing learning activity 

Removing a Learning Path from a competence 

Removing a Learning Activity from a learning path 

Removing a Knowledge Asset from an learning activity 

Macro-Level SRL Process: Evaluation & Reflection 

Micro-Level SRL Process 

SRL Events in Learn-B 

Evaluation 

Rating a Learning Path, Learning Activity or a Knowledge Asset 

Marking a Learning Goal, Competence, or Learning Activity as "completed" 

Leaving a comment for a Competence, Learning Path, Learning Activity or 
Knowledge Asset 

Adding new keywords to or updating existing keywords of a Learning Goal, 
competence, Learning Path, Learning Activity or Knowledge Asset 

Reflection 

Leaving a comment for a Competence, Learning Path, Learning Activity or 
Knowledge Asset 

Adding new keywords to or updating existing keywords of a Learning Goal, 
competence, Learning Path, Learning Activity or Knowledge Asset 

Updating the visibility property of Learning Goal, competence, Learning Path, 
Learning Activity or Knowledge Asset 

Sharing a Learning Goal with a recommended colleague 

Recommending a Learning Goal to a colleague 

Intervention Events 

Intervention 1: Providing Usage Information 

Intervention Feature 

Intervention Events in Learn-B 

Analytics 

Clicking on the Achievement tab (under Analytics) of an available 

Competence, Learning Path or Learning Activity 

Clicking on Duties node (the summary tab will show in the right panel) 
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Social Stream 

Clicking on the Social Wave tab (under Analytics) of an available Competence, 
Learning Path, Learning Activity or Knowledge Asset 

Social Stand 

Clicking on the comments tab of a Competence, Learning Path , Learning 
Activity or Knowledge Asset 

Clicking on the data tab of a Competence, Learning Path, Learning Activity or 
Knowledge Asset 

Intervention II: Social Wave 

Intervention Feature 

Intervention Events in Learn-B 

Generic Social Wave 

Clicking on one's Social Wave tab 

Learning Resources' Social Waves 

Clicking on the Social Wave tab of one's Learning Goal, Competence, Learning 
Path, Learning Activity or Knowledge Asset 

Bubble Social Waves 

Clicking on the Social Wave Bubbles tab (under Analytics) of an available 
Competence, Learning Path, Learning Activity or Knowledge Asset 

Clicking on Duties, Roles, Tasks or Projects folder 

Clicking on a single Duty under the Duties folder 

Clicking on a single Role under the Roles folder 

Clicking on single Task under the Tasks folder 

Clicking on single Project under the Projects folder 

Intervention III: Progress-o-meters 


Clicking on the Goal-o-meter tab (under Analytics) of one's Learning Goal 


Clicking on the Competence-o-meter tab (under Analytics) of one's 

Competence 


Clicking on the Progress-o-meter tab (under Analytics) of a Learning Path 


Clicking on the Progress-o-meter tab (under Analytics) of a Learning Activity 

Intervention IV: User-recommended Learning Goals 


Clicking on a single Learning Goal under the Recommended Learning Goals 

folder 

Intervention V: Recommended available Competences 


Clicking on different Competences related to a Duty, Role, Task, or Project 


Clicking on Users who are acquiring/have already acquired an available 

competence 

Intervention VI: Recommended available Learning Paths, Learning Activities, and Knowledge Assets 
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Clicking on a Learning Path for an available competence 


Clicking on a Learning Activity within an available learning path 


Clicking on a Knowledge Asset related to an available learning activity 


Clicking on a recommended Learning Path 


Clicking on an abandoned Learning Path, i.e. a previously chosen 
recommended learning path 


Clicking on the data tab of an available Learning Path, Learning Activity, or 
Knowledge Asset 

Intervention VII: Knowledge sharing Profiles 


Clicking on one's Analytics tab (the Knowledge Sharing Profiles tab is the only 
tab, so will open automatically) 
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